Imaging control system, control apparatus, control method, and storage medium

ABSTRACT

A control apparatus for controlling an imaging apparatus capable of changing an imaging direction of an imaging unit in panning and tilting directions includes an acquisition unit configured to acquire a captured image captured by the imaging unit, and an provision unit configured to provide, together with the captured image, when the imaging direction of the imaging unit coincides with a panning rotational axis for rotating the imaging direction in the panning direction, a panning direction guide for indicating that changing the imaging direction in the panning direction rotates the captured image around an intersection of an optical axis of the imaging unit and a plane containing the captured image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging control system forcontrolling an imaging apparatus, an apparatus and a method forcontrolling the imaging apparatus, and a storage medium.

2. Description of the Related Art

A conventional imaging control system includes an imaging apparatusconnected to a network and a client apparatus which controls the imagingapparatus via the network. The imaging control system can controlpanning and tilting drive mechanisms of the imaging apparatus throughoperations on the client apparatus. A known imaging control system isdirected, when driving an imaging apparatus for panning and tilting, tomaking it easier for a user to orient the imaging apparatus in thedirection of a target object (for example, Japanese Patent ApplicationLaid-Open No. 2004-266876). The imaging control system stores in amemory of an imaging apparatus a table for setting a relation betweenthe panning angle and name display characters such as “North”, “South”,“East”, and “West.” The imaging control system orients the imagingapparatus to a panning angle corresponding to a name display characterinput by a user, and displays on a monitor the name display charactercorresponding to the direction to which the imaging apparatus isoriented.

A known method displays buttons for indicating the panning and tiltingdirections at predetermined positions around a captured image (forexample, Japanese Patent Application Laid-Open No. 4-373386 and JapanesePatent Application Laid-Open No. 8-237533). However, there has been acase where the conventional method cannot suitably show a user theimaging direction since the panning and tilting directions dynamicallychange in response to operations of the panning, tilting, and rotationmechanisms.

As a method for assisting the user to orient the imaging direction ofthe imaging apparatus to a target direction when driving the imagingapparatus for panning and tilting, the panning and tilting drivedirections may be possibly superimposed onto the captured image, asillustrated in FIG. 9A. When the user changes the imaging direction ofthe imaging apparatus to the panning or tilting direction, the methodillustrated in FIG. 9A superimposes onto the captured image as directionguides 913 and 914 a moving path to be traced by an imaging center 911which is an intersection of a plane containing the captured image and anoptical axis 901 of the imaging apparatus. However, when superimposingthe panning and tilting drive directions, there arises a problem thatthe user cannot recognize the panning and tilting drive directions in aspecific imaging direction in some cases.

A problem arising in drive direction display will be described below.Modes for displaying the drive direction on a display image and a changein panning direction display accompanying the operation of the tiltingmechanism will be described below with reference to FIGS. 9A and 9B.

Referring to FIG. 9A, the optical axis 901 denotes an imaging directionof an imaging apparatus, a display image 910 denotes an image displayedon a display apparatus, an imaging center 911 denotes a center of thecaptured image, and a panning rotation center 912 denotes the positionof a panning rotation center on the display image 910. Referring to FIG.9A, a panning direction guide 913 is a panning direction guide forindicating the panning drive direction by the panning drive mechanism,and a tilting direction guide 914 is a tilting direction guide forindicating the tilting drive direction by the tilting drive mechanism.Each of the panning direction guide 913 and the tilting direction guide914 is represented by a straight line or curve passing through theimaging center 911. Each of the direction guides 913 and 914 dynamicallychanges in shape with the changing imaging direction, i.e., according tooperations of the panning, tilting, and rotation mechanisms.

As illustrated in FIG. 9A, the closer the optical axis 901 is to theperpendicular to the panning rotational axis 203, the straighter thepanning direction guide 913 and the smaller its curvature. FIG. 9Billustrates a display image 920 produced when the tilting mechanism isoperated from the state of the optical axis 901 to bring the imagingdirection close to the panning rotational axis 203. As illustrated inFIG. 9B, the closer the imaging direction is to the panning rotationalaxis 203, the shorter the distance between the imaging center 911 andthe panning rotation center 912 and the larger the curvature of thepanning direction guide 913 on the display image 920. When the imagingdirection is close to the panning rotational axis 203, the panningdirection guide 913 forms a circle or arc having a small radius, makingit difficult for the user to recognize the panning drive direction. Ifthe imaging apparatus 110 continues the tilting operation from theabove-mentioned state, the panning direction guide 913 converges to apoint and can no longer be displayed when the imaging directioncoincides with the panning rotational axis 203. In this state, the usercannot recognize the panning drive direction.

As mentioned above, when the panning direction guide 913 and the tiltingdirection guide 914 are superimposed onto the captured image such thatthese direction guides pass through the center of the captured image, itbecomes difficult or impossible for the user to recognize the panningmoving direction. In this state, the user may be confused whenattempting to adjust the orientation of the imaging apparatus 110 with adesired imaging direction.

SUMMARY OF THE INVENTION

The present invention is directed to making it easier for a user torecognize drive directions of an imaging apparatus when attempting toadjust the orientation of the imaging apparatus with a desired imagingdirection.

According to an aspect of the present invention, a control apparatus forcontrolling an imaging apparatus capable of changing an imagingdirection of an imaging unit in panning and tilting directions includesan acquisition unit configured to acquire a captured image captured bythe imaging unit, and a provision unit configured to provide, togetherwith the captured image, when the imaging direction of the imaging unitcoincides with a panning rotational axis for rotating the imagingdirection in the panning direction, a panning direction guide forindicating that changing the imaging direction in the panning directionrotates the captured image around an imaging center which is anintersection of an optical axis of the imaging unit and a planecontaining the captured image.

According to an exemplary embodiment of the present invention,superimposing a panning direction guide not overlapping an imagingcenter onto a captured image makes it easier for the user to recognizedrive directions of the imaging apparatus.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating an imaging control systemaccording to first and second exemplary embodiments of the presentinvention.

FIGS. 2A and 2B illustrate an imaging apparatus having panning, tilting,and rotation mechanisms.

FIGS. 3A to 3F illustrate display images according to the firstexemplary embodiment.

FIG. 4 illustrates a display image produced when an imaging apparatusaccording to the first exemplary embodiment is driven for rotation.

FIG. 5 is a flow chart illustrating operations of a control apparatusaccording to the first exemplary embodiment.

FIG. 6 is a flow chart illustrating operations of the imaging apparatusaccording to the first exemplary embodiment.

FIGS. 7A and 7B illustrate a virtual spherical surface around theimaging apparatus.

FIGS. 8A and 8B illustrate display images according to the secondexemplary embodiment.

FIGS. 9A and 9B illustrate an exemplary conventional imaging controlsystem.

FIG. 10 is a block diagram illustrating an imaging control systemaccording to third and fourth exemplary embodiments of the presentinvention.

FIGS. 11A to 11D illustrate display images according the third exemplaryembodiment of the present invention.

FIGS. 12A to 12C illustrate display images according to the fourthexemplary embodiment of the present invention.

FIGS. 13A and 13B illustrate display images produced when an imagingapparatus according to the fourth exemplary embodiment is driven forrotation.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

FIG. 1 is a block diagram illustrating an imaging control systemaccording to a first exemplary embodiment of the present invention.Referring to FIG. 1, an imaging apparatus 110 captures an image. Aclient apparatus 130 sets up the imaging apparatus 110, and displays andrecords the captured image. A network 190 connects the imaging apparatus110 to the client apparatus 130. The network 190 includes a plurality ofrouters, switches, and cables satisfying, for example, Ethernet andother communications standards. The present exemplary embodiment isbased on any communication standard, scale, and configuration as long ascommunications between respective servers and clients are achieved. Forexample, the network 190 may be composed of any one of the Internet, awired local area network (LAN), a wireless LAN, and a wide area network(WAN).

A configuration of the imaging apparatus 110 according to the presentexemplary embodiment will be described below. An imaging unit 111includes an image sensor and an optical system for forming a subjectimage on the image sensor, and is configured to capture the image formedon the image sensor with an intersection of the optical axis of theoptical system and the image sensor as an imaging center. The imagesensor is a complementary metal-oxide semiconductor (CMOS) sensor or acharge-coupled device (CCD) sensor.

A panning drive unit 112, a tilting drive unit 113, and a rotation driveunit 114 drive the imaging unit 111 based on a command from a centralprocessing unit (CPU) 115 (described below). The panning drive unit 112drives the imaging unit 111 for panning. The tilting drive unit 113drives the imaging unit 111 for tilting. The rotation drive unit 114rotates a lens unit 207 (described below) constituting the imaging unit111, around a rotational axis for rotation (described below). In thepresent exemplary embodiment, the rotational axis for rotation coincideswith an imaging direction 208 of the imaging unit 111. The imaging unit111 may change the imaging direction 208 to the panning, tilting, orrotation direction with manual operation.

A video processing unit 116 processes a video signal captured by theimaging unit 111. The video signal processing performed by the videoprocessing unit 116 includes converting the video signal captured by theimaging unit 111 into digital form, and coding the digital video signal.The digital video signal format may be, for example, the YUV format. Thedigital video signal coding method may be based on any one of the MPEG4,H.264, MJPEG, and JPEG standards. The video signal having undergone thevideo signal processing by the video processing unit 116 is once storedin a memory 117 and then output to the client apparatus 130 via acommunication control unit 118 and the network 190, under control of theCPU 115 (described below).

The communication control unit 118 applies packetization processing tothe coded digital video signal. To output the digital video signal tothe network 190, the communication control unit 118 performs packetmultiplexing processing based on a predetermined format, under controlof the CPU 115. The predetermined format includes the hypertext transferprotocol (HTTP) and the real-time transport protocol (RTP). Thecommunication control unit 118 transmits to the network 190 the digitalvideo signal captured by the imaging unit 111.

The CPU 115 loads a program from the memory 117 and then executes it tocontrol operations of each component included in the imaging apparatus110. The CPU 115 also controls the panning drive unit 112, the tiltingdrive unit 113, and the rotation drive unit 114 according to commandsreceived from the client apparatus 130 via the network 190. Based onresults of driving the imaging apparatus 110 by the panning drive unit112, the tilting drive unit 113, and the rotation drive unit 114, theCPU 115 also transmits panning, tilting, and rotation positions of theimaging unit 111 to the client apparatus 130 via the communicationcontrol unit 118. The CPU 115 acquires from the imaging unit 111 aviewing angle (angle of view) of the imaging unit 111 and the size ofthe captured image captured by the imaging unit 111, and transmits themto the client apparatus 130. The size of the image refers to, forexample, the length of the diagonal line of the captured image 302.

An exemplary imaging apparatus 110 is illustrated in FIGS. 2A and 2B.FIG. 2A is a side view of the imaging apparatus 110. Referring to FIG.2A, a rotation unit for panning includes a bottom case 201 and aturntable 202, which rotates around the panning rotational axis 203. Thepanning rotational axis 203 rotates the imaging direction 208 in thepanning direction. The panning drive unit 112 includes a stepping motorand is configured to drive the imaging unit 111 for panning by rotatingthe turntable 202.

FIG. 2B is a front view of the imaging apparatus 110. A rotation unitfor tilting includes a lens support 204 and a lens case 206, and isconfigured to rotate the lens case 206 around a tilting rotational axis205. The tilting drive unit 113 includes a stepping motor and isconfigured to drive the imaging unit 111 for tilting by rotating thelens case 206. A rotation unit for rotation includes the lens case 206and the lens unit 207, which rotates around the rotational axis forrotation. The rotation drive unit 114 includes a stepping motor and isconfigured to drive the imaging unit 111 for rotation by rotating thelens unit 207.

The client apparatus 130 will be described below with reference toFIG. 1. Under control of a CPU 137, the client apparatus 130 receivesthe captured image transmitted from the imaging apparatus 110 via acommunication control unit 131, stores the captured image in a memory132, and then transmits it to a display control unit 133.

The display control unit 133 superimposes a panning direction guide forindicating the panning drive direction by the panning drive unit 112onto the captured image captured by the imaging unit 111. The displaycontrol unit 133 also superimposes a tilting direction guide forindicating the tilting drive direction by the tilting drive unit 113onto the captured image captured by the imaging unit 111. Then, thedisplay control unit 133 superimposes onto the captured image a panningdirection guide for indicating the panning drive direction by thepanning drive unit 112 and a tilting direction guide for indicating thetilting drive direction by the tilting direction guide 113, and displaysthe resultant image on a display apparatus 170. Display modes will bedescribed below with reference to FIGS. 3A to 3F. Then, the displaycontrol unit 133 performs display control for displaying user interfacessuch as operation buttons on the display apparatus 170.

A user interface (UI) control unit 136 receives a command from an inputapparatus 150 (described below), and transmits to the imaging apparatus110 commands for controlling the panning drive unit 112, the tiltingdrive unit 113, and the rotation drive unit 114 via the communicationcontrol unit 131, under control of the CPU 137. The UI control unit 136includes a panning operation unit 138 for operating the panning driveunit 112, a tilting operation unit 139 for operating the tilting driveunit 113, and a rotation operation unit 140 for operating the rotationdrive unit 114.

The input apparatus 150 inputs from the user various commands foroperating the imaging apparatus 110. The input apparatus 150 outputs tothe UI control unit 136 panning, tilting, and rotation drive commandsinput by the user. The input apparatus 150 is, for example, a keyboard,mouse, etc. The user operates the operation buttons displayed on thedisplay apparatus 170 (described below) by using the keyboard or mouseto operate the imaging apparatus 110. The input apparatus 150 and thedisplay apparatus 170 may be configured as one apparatus, and commandsto the imaging apparatus 110 may be input from a touch panel.

Through display control by the display control unit 133, the displayapparatus 170 displays the captured image captured by the imagingapparatus 110 and direction guides indicating the panning and tiltingdrive directions. Although FIG. 1 illustrates the client apparatus 130and the display apparatus 170 as independent apparatuses, they may beconfigured as one apparatus. For example, when using a personal computer(PC) as the client apparatus 130, displaying the captured image anddirection guides on the display of the PC enables the PC to serve alsoas the display apparatus 170.

Exemplary display images displayed on the display apparatus 170 throughdisplay control by the display control unit 133 are illustrated in FIGS.3A to 3F. FIG. 3A illustrates a state where the imaging direction 208 isclose to the perpendicular to the panning rotational axis 203. Referringto FIG. 3A, a display image 310 is displayed on the display apparatus170 after the display control unit 133 applies display control to thecaptured image received from the imaging apparatus 110 by the clientapparatus 130. An imaging center 311 is an intersection of the opticalaxis of the optical system for forming a subject image on the imagesensor and a plane containing the captured image captured by the imagingunit 111.

A panning rotation center 312 indicates the position of a panningrotation center in the plane of the display image 310. A panning mainguide 313 is a panning direction guide for indicating the panning movingdirection such that the panning main guide 313 overlaps a moving path tobe traced by the imaging center 311 while the imaging unit 111 is drivenfor panning. A method for setting the panning rotation center 312 and amethod for displaying the panning main guide 313 will be described belowwith reference to FIG. 3E. A tilting main guide 314 is a tiltingdirection guide for indicating the tilting moving direction such thatthe tilting main guide 314 overlaps a moving path to be traced by theimaging center 311 while the imaging unit 111 is driven for tilting. Apanning sub guide 315 is a panning direction guide which does not passthrough the imaging center 311. In the present exemplary embodiment, thepanning sub guide 315 is superimposed onto the captured image such thatthe panning sub guide 315 does not overlap a moving path to be traced bythe imaging center 311 while the imaging unit 111 is driven for panning.The panning sub guide 315 is displayed on the outer side of the panningmain guide 313 at a suitable interval from the panning rotation center312. The panning sub guide 315 shows the user the panning drivedirection similarly to the panning main guide 313. Thus, the displaycontrol unit 133 superimposes onto the captured image the panningdirection guides not passing through the imaging center 311 when theimaging unit 111 is driven for panning. A method for displaying thepanning sub guide 315 will be described below.

A method for displaying the panning main guide 313 and a method forsetting the panning rotation center 312 will be described below withreference to FIG. 3E. Referring to FIG. 3E, the panning rotational axis203 is used by the imaging unit 111 for panning rotation, and an imagingapparatus position 300 indicates the position of the imaging apparatus110. Referring to FIG. 3E, a captured image 302 is virtually arranged soas to perpendicularly intersect with the imaging direction 208 of theimaging unit 111, and a vector 301 indicates the direction of the centerof the captured image 302. The magnitude of the vector 301 is set upaccording to the viewing angle of the captured image 302 and the size ofthe captured image 302.

The display control unit 133 performs display control for the panningmain guide 313 and sets up the panning rotation center 312. The displaycontrol unit 133 first calculates the magnitude of the vector 301 basedon the viewing angle acquired from the imaging apparatus 110 and thesize of the captured image 302. The magnitude of the vector 301 iscalculated, for example, by formula (1).L=d/tan α  (1)where L denotes the magnitude of the vector 301, 2d denotes the lengthof a diagonal line of the captured image 302, and 2α denotes the viewingangle of a diagonal line of the captured image 302.

The display control unit 133 also calculates the value of an angle 303between the panning rotational axis 203 and the vector 301 based on atilting angle acquired from the imaging apparatus 110. Then, the displaycontrol unit 133 derives the length of a panning rotation radius 304based on the calculated magnitude L of the vector 301 and the value ofthe angle 303. The length of the panning rotation radius 304 can becalculated by formula (2).r=L·sin β  (2)where r denotes the length of the panning rotation radius 304 and βdenotes the value of the angle 303.

Based on the thus-obtained panning rotation radius 304, the displaycontrol unit 133 calculates the circumference of a circleperpendicularly intersecting with the panning rotational axis 203 as apanning rotational orbit 305. Then, the display control unit 133projects the panning rotational orbit 305 (viewed from the imagingapparatus position 300) from the imaging apparatus position 300 onto thevirtual captured image 302 to display the panning main guide 313.

Then, the display control unit 133 sets up an intersection of a planecontaining the captured image 302 and the panning rotational axis 203 asthe panning rotation center 312. In the present exemplary embodiment,when the value of the angle 303 is 90 degrees, the panning rotationcenter 312 is to be set as a point existing above in FIG. 3E, on astraight line passing through the imaging center 311 of the capturedimage 302 and being parallel to the panning rotational axis 203 on theplane containing the captured image 302. Setting the panning rotationcenter 312 when the value of the angle 303 is 90 degrees is based notonly on the above-mentioned method. When the value of the angle 303 is90 degrees, the panning rotation center 312 may not to be set on theplane containing the captured image 302.

Displaying the panning main guide 313 and setting the panning rotationcenter 312 are based not only on the above-mentioned methods. Forexample, the panning rotation center 312 is set to a point existingabove in FIG. 3A in the tilting drive direction, where the distance fromthe imaging center 311 of the display image 310 is equal to the panningrotation radius 304. An arc around the panning rotation center 312 andhaving a radius equal to the panning rotation radius 304 may bedisplayed as the panning main guide 313. Further, the panning main guide313 may not be displayed as an arc. For example, as illustrated in FIG.3F, the panning main guide 313 may be displayed as arrows to indicatethe panning drive direction. For example, arrows may be drawn to becircumscribed on a curve drawn by projecting the panning rotationalorbit 305. Alternatively, arrows may be drawn such that their respectiveends are inscribed into a curve drawn by projecting the panningrotational orbit 305. The above-mentioned methods for displaying thepanning drive direction are to be considered as examples for indicatingthe panning drive direction, and displaying the panning drive directionis based not only thereon. In the above-mentioned examples, although thepanning main guide 313 is displayed as a part or the whole of a circlehaving an equal radius, the panning main guide 313 may be drawn as apart or the whole of an ellipse around the imaging center 311.

A method for displaying the panning sub guide 315 will be describedbelow. The panning sub guide 315 can be displayed to be constantlydistant from the panning main guide 313. Alternatively, similar to thepanning main guide 313, the panning drive direction may be displayed asarrows inscribed or circumscribed with respect to a curve drawn to beconstantly distant from the panning main guide 313. Although the panningmain guide 313 is drawn by a solid line and the panning sub guide 315 isdrawn by a dashed line in FIG. 3A, the panning main guide 313 may bedrawn by a dashed line and the panning sub guide 315 may be drawn by asolid line. Displaying the panning sub guide 315 is based not only onthe above-mentioned method but also on any other method as long as themethod indicates the panning drive direction such that the method doesnot pass through the imaging center 311 when the imaging unit 111 isdriven for panning.

Suppose that, in the state illustrated in FIG. 3A, the user inputs fromthe input apparatus 150 a command for driving the imaging unit 111 fortilting, the imaging unit 111 is driven for tilting toward the panningrotation center 312 (in the upward direction in FIG. 3A), and theimaging direction 208 comes close to the panning rotational axis 203. Adisplay image 320 displayed in this state will be described below withreference to FIG. 3B. When the imaging direction 208 comes close to thepanning rotational axis 203, the imaging center 311 and the panningrotation center 312 come close to each other to reduce the radius of thepanning main guide 313, making it difficult for the user to recognizethe panning drive direction. However, displaying the panning sub guide315 on the outer side of the panning main guide 313 enables indicatingthe panning drive direction with an arc having a larger radius, makingit easier for the user to recognize the panning drive direction.

Suppose that, in the state illustrated in FIG. 3B, the imaging unit 111is further driven for tilting toward the panning rotation center 312,and the imaging center 311 coincides with the panning rotational axis203. A display image 330 in this state will be described below withreference to FIG. 3C. In this state, since the position of the panningrotation center 312 derived from state values of the imaging apparatus110 coincides with the imaging center 311, the panning main guide 313converges to a point and can no longer be displayed. However, thepanning sub guide 315 set up to be displayed on the outer side of thepanning main guide 313 is kept being displayed without converging to apoint, enabling the user to recognize the panning drive direction. Thus,the display control unit 133 provides, together with a captured image, apanning direction guide indicating that changing the imaging direction208 to the panning direction rotates the captured image around theimaging center 311 when the imaging direction 208 of the imaging unit111 coincides with the panning rotational axis 203.

Further, like a display image 340 illustrated in FIG. 3D, the displaycontrol unit 133 can display a second panning sub guide 316 on the innerside of the panning main guide 313. The second panning sub guide 316 canshow the user in advance how the panning main guide 313 will change whenthe imaging unit 111 is driven for tilting toward the panning rotationcenter 312 (in the upward direction in FIG. 3D). Therefore, even if thepanning main guide 313 converges to a point and can no longer bedisplayed as mentioned above, the second panning sub guide 316 cansuitably inform the user of the moving direction of the captured imagewithout confusing the user. Thus, the imaging control system or thecontrol apparatus according to the present exemplary embodiment can makeit easier for the user to recognize the panning drive direction of theimaging apparatus 110.

Suppose that, in the state illustrated in FIG. 3D, the imaging unit 111is driven for rotation. A display image 410 produced in this state isillustrated in FIG. 4. When the imaging unit 111 is driven for rotation,the display image 340 in FIG. 3D rotates around the imaging center 311.With the rotation of the display image 340, the display control unit 133rotates the orientation of the panning main guide 313, the tilting mainguide 314, and the panning sub guides 315 and 316 around the imagingcenter 311, and superimposes them onto the captured image. Thus, thedisplay control unit 133 superimposes the panning direction guides andthe tilting direction guides on the captured image depending on theamount of rotation drive operation by the rotation drive unit 114.

Thus, the client apparatus 130 can show the user the panning and tiltingdrive directions when the imaging unit 111 is driven for rotation.Further, even when the panning rotation center 312 coincides with theimaging center 311 and the panning main guide 313 converges to a pointand can no longer be displayed, the panning sub guide 315 can show theuser the panning drive direction. Then, the panning sub guide 316 canshow the user in advance how the panning main guide 313 will change whenthe imaging unit 111 is driven for tilting toward the panning rotationcenter 312. When the imaging unit 111 is driven for rotation, thepanning and tilting drive directions incline as illustrated in FIG. 4,making it difficult for the user to grasp the panning and tilting drivedirections. However, the present exemplary embodiment can show the userthe panning and tilting drive directions even when the optical axis ofthe imaging apparatus 110 comes close to the panning rotational axis203.

Thus, the panning direction guides such as the panning main guide 313and the panning sub guides 315 and 316 indicate that changing theimaging direction 208 to the panning direction rotates a predeterminedpoint on the captured image around the panning rotation center 312.Although the panning direction guides indicate a part or the whole of acircle having a constant radius in the above-mentioned examples forconvenience, they may indicate a part or the whole of an ellipse aroundthe imaging center 311.

Operations of the client apparatus 130 according to the presentexemplary embodiment will be described below with reference to the flowchart in FIG. 5. The processing of the flow chart in FIG. 5 isimplemented by a program for causing the client apparatus 130 to executethe processing in FIG. 5. The client apparatus 130 includes a computer(the CPU 137) for executing a program loaded from the memory 132included in the client apparatus 130.

In step S90, the communication control unit 131 of the client apparatus130 acquires the captured image captured by the imaging unit 111. Instep S100, the client apparatus 130 enters the panning, tilting, androtation (PTR) setting mode. The PTR setting mode is a mode whichenables the user to set up operations of the panning drive unit 112, thetilting drive unit 113, and the rotation drive unit 114 of the imagingapparatus 110 from the client apparatus 130. More specifically, when thePTR mode is entered, the display control unit 133 displays on thedisplay apparatus 170 the display image 340 and operation buttons forsetting panning, tilting, and rotation operations upon reception of acommand from the CPU 137. Then, the client apparatus 130 issues acommand to the imaging apparatus 110 according to a command input by theuser by operating the operation buttons on the input apparatus 150.

In step S110, the client apparatus 130 displays the panning main guide313, the tilting main guide 314, and the panning sub guides 315 and 316according to the current panning, tilting, and rotation angles, asillustrated in FIGS. 3A to 3F and 4.

In step S120, the client apparatus 130 determines whether a command fordriving the imaging apparatus 110 is input from the input apparatus 150.When a command for driving the imaging apparatus 110 is input from theinput apparatus 150 to the client apparatus 130 (YES in step S120), theprocessing proceeds to step S130. In step S130, the client apparatus 130transmits to the imaging apparatus 110 a drive command corresponding toan operation input by the user. Then, the processing returns to stepS110.

Operations of the imaging apparatus 110 according to the presentexemplary embodiment will be described below with reference to the flowchart in FIG. 6. The processing of the flow chart in FIG. 6 isimplemented by a program for causing the imaging apparatus 110 toexecute the processing illustrated in FIG. 6. The imaging apparatus 110includes a computer (the CPU 115) for executing a program loaded fromthe memory 117 included in the imaging apparatus 110.

In step S200, the imaging apparatus 110 initializes the imagingapparatus 110, i.e., the panning drive unit 112, the tilting drive unit113, and the rotation drive unit 114, as well as the imaging unit 111.Through initialization, the imaging apparatus 110 confirms originpositions of the panning drive unit 112, the tilting drive unit 113, andthe rotation drive unit 114, and detects current imaging positions ofthese units.

In step S210, the imaging apparatus 110 transmits to the clientapparatus 130 current positions of the panning drive unit 112, thetilting drive unit 113, and the rotation drive unit 114, as well as theviewing angle of the captured image as a status. Upon completion ofinitialization, in step S220, the imaging apparatus 110 transmits thecaptured image to the client apparatus 130. In step S230, the imagingapparatus 110 waits for reception of a command from the client apparatus130.

When a command is received from the client apparatus 130 (YES in stepS230), the processing proceeds to step S240. In step S240, the imagingapparatus 110 drives the imaging unit 111 for panning, tilting, orrotation, or a plurality of drive operations according to the receivedcommand. The panning drive unit 112, the tilting drive unit 113, and therotation drive unit 114 drive the imaging unit 111 for panning, tilting,and rotation, respectively. In step S250, after the imaging unit 111 isdriven as specified by the client apparatus 130, the imaging apparatus110 transmits to the client apparatus 130 current positions of thepanning drive unit 112, the tilting drive unit 113, and the rotationdrive unit 114, as well as the viewing angle of the captured image as astatus. Then, the processing returns to step S230 to wait for commandreception.

Although the panning sub guides 315 and 316 are constantly displayed onthe outer and inner sides of the panning main guide 313, respectively,the timing of displaying the panning sub guides 315 and 316 is notlimited to thereto. The panning sub guides 315 and 316 may be displayedonly when the imaging area of the imaging unit 111 falls within aspecific range. For example, when the value of the angle 303 between thepanning rotational axis 203 used by the panning drive unit 112 to drivethe imaging unit 111 for panning and the vector 301 of the imaging unit111 is equal to or less than a threshold value, the display control unit133 can superimpose the panning sub guides 315 and 316 onto the capturedimage. In this case, the vector 301 coincides with the optical axis ofthe imaging unit 111. Alternatively, when the distance between theimaging center 311 and the panning rotation center 312 illustrated inFIG. 3A is equal to or less than a predetermined value, the panning subguides 315 and 316 can be displayed.

As mentioned above, the distance between the imaging center 311 and thepanning rotation center 312 is determined by the length of the panningrotation radius 304 illustrated in FIG. 3E. The length of the panningrotation radius 304 depends on the viewing angle of the imaging unit 111and the size of the captured image 302. Therefore, changing thethreshold value of the angle 303 according to the viewing angle of theimaging unit 111 and the size of the captured image enables the displaycontrol unit 133 to display the panning sub guides 315 and 316 when thedistance between the imaging center 311 and the panning rotation center312 is equal to or less than the predetermined value. Determiningwhether the imaging area of the imaging unit 111 falls within a specificrange is based not only on the above-mentioned method but also on anyother method.

Processing is based not only on the above-mentioned modes. The panningsub guide 315 may indicate that changing the imaging direction 208 tothe panning direction rotates the captured image around the imagingcenter 311 when the imaging direction 208 of the imaging unit 111coincides with the panning rotational axis 203. A case where the imagingdirection 208 of the imaging unit 111 coincides with the panningrotational axis 203 is not limited to a case where they actuallycoincide with each other. For example, such cases include a case where,even when the panning main guide 313 is displayed, the user cannotrecognize that changing the imaging direction 208 to the panningdirection rotates the captured image since the angle between the imagingdirection 208 of the imaging unit 111 and the panning rotational axis203 is equal to or less than a threshold value. Such cases include acase where the panning main guide 313 cannot be displayed since theangle between the imaging direction 208 of the imaging unit 111 and thepanning rotational axis 203 is equal to or less than the thresholdvalue. The threshold value can be arbitrarily set, for example, to 5degrees.

When the imaging direction 208 of the imaging unit 111 coincides withthe panning rotational axis 203, processing is not limited tosuperimposition of the panning sub guide 315 onto the captured image asillustrated in FIG. 3C, but a part or the whole of a circle may bedisplayed around the captured image. Thus, it becomes possible toindicate that the captured image will rotate around the imaging center311.

When the imaging direction 208 of the imaging unit 111 coincides withthe panning rotational axis 203, a text message may be displayed on thedisplay screen to notify the user that changing the imaging direction208 to the panning direction rotates the captured image, instead ofdisplaying the panning sub guide 315 as illustrated in FIG. 3C. In thiscase, a text message such as “changing the imaging direction to thepanning direction rotates the screen” may be displayed together with thecaptured image. Alternatively, when the imaging direction 208 of theimaging unit 111 coincides with the panning rotational axis 203, a textmessage may be displayed on the display screen to enable the user torecognize the direction of screen rotation accompanying the panningoperation. In this case, a text message such as “panning to the rightrotates the screen counterclockwise” or “panning to the left rotates thescreen clockwise” may be displayed together with the captured image.

When the imaging direction 208 of the imaging unit 111 coincides withthe panning rotational axis 203, an audio message may be generated tonotify the user that changing the imaging direction 208 to the panningdirection rotates the captured image around the imaging center 311. Forexample, when the imaging direction 208 of the imaging unit 111coincides with the panning rotational axis 203, an audio guidance suchas “the captured image will rotate” may be provided when panningoperation is made. Similar to displaying the above-mentioned textmessages, an audio message may notify the user of the direction ofscreen rotation accompanying the panning operation.

Thus, even after the panning main guide 313 is no longer displayed, thepanning sub guide 315 can notify the user of the panning drivedirection. Further, when the panning main guide 313 is highly likely toconverge to the panning rotation center 312 when the imaging unit 111 isdriven for tilting, the panning sub guide 316 may be displayed on theinner side of the moving path of the imaging center 311 (on the side ofthe panning rotation center 312) to notify the user that continuing thetilting drive operation causes the panning main guide 313 to be hiddenby the panning sub guide 316. Then, hiding the panning sub guides 315and 316 when the imaging area of the imaging unit 111 is not within aspecific range enables giving priority to the viewability of thecaptured image. The user may not only use a threshold value but alsodisplay and hide the panning sub guides 315 and 316. In the presentexemplary embodiment, displaying the panning and tilting movingdirections is based not only on the above-mentioned method.

According to the above-mentioned configuration, the display control unit133 superimposes onto the captured image a panning direction guide notoverlapping a moving path to be traced by the imaging center 311 whilethe imaging unit 111 is driven for panning. Therefore, the user canrecognize the moving direction even when the imaging center 311 overlapsthe panning rotation center 312 and the panning main guide 313 is nolonger displayed. Further, displaying the panning sub guide 316 on theinner side of the moving path of the imaging center 311 (on the side ofthe panning rotation center 312) enables the user to recognize how thepanning main guide 313 changes before the imaging center 311 overlapsthe panning rotation center 312. Thus, the present exemplary embodimentcan suitably notify the user of the moving direction of the imagingarea. Therefore, the user can recognize the panning drive direction ofthe imaging apparatus 110 more easily than in a case where only thepanning main guide 313 is displayed on a display image.

Although the present exemplary embodiment displays the panning mainguide 313 indicating the panning moving direction such that the panningmain guide 313 overlaps a moving path to be traced by the imaging center311, the effect of the present invention can be achieved even bydisplaying only the panning sub guides 315 and 316 without displayingthe panning main guide 313.

A second exemplary embodiment of the present invention will be describedbelow focusing on an imaging control system for superimposing onto thecaptured image the latitude and meridian lines on a spherical surfacevirtually set up around the imaging apparatus 110.

A configuration of the imaging control system according to the secondexemplary embodiment will be described below. In addition to the panningmain guide 313 and the tilting main guide 314, the display control unit133 of the client apparatus 130 superimposes onto the captured image thelatitude and meridian lines on the spherical surface virtually set uparound the imaging apparatus 110. Other elements are similar to those ofthe imaging control system according to the first exemplary embodimentillustrated in FIG. 1, and duplicated explanation will be omitted.Elements similar to those in the first exemplary embodiment illustratedin FIG. 1 are assigned the same reference numerals.

Display images displayed on the display apparatus 170 through displaycontrol by the display control unit 133 will be described below withreference to FIGS. 7A, 7B, 8A, and 8B. FIG. 7A illustrates a virtualspherical surface virtually set up around the imaging apparatus 110.Referring to FIG. 7A, an imaging apparatus position 710 indicates aposition at which the imaging apparatus 110 is installed. The panningrotational axis 203 is used to drive the imaging unit 111 for panning. Aspherical surface 720 is virtually set up around the imaging apparatus110. The spherical surface 720 is provided with latitude lines 721 andmeridian lines 722 at predetermined positions thereon. In the presentexemplary embodiment, the latitude lines 721 are set up at fixedintervals from the equatorial plane around the imaging apparatusposition 710. The meridian lines 722 are set up at fixed intervals froma predetermined reference meridian line around the imaging apparatusposition 710. Setting the latitude lines 721 and the meridian lines 722is based not only on the above-mentioned method.

Exemplary display images displayed on the display apparatus 170 when theimaging apparatus 110 captures an image at the imaging apparatusposition 710 will be described below with reference to FIGS. 8A and 8B.The imaging apparatus 110 transmits the captured image captured therebyto the client apparatus 130. Then, the display control unit 133superimposes as panning direction guides the latitude lines 721 onto thecaptured image arranged on the virtual spherical surface 720. In thepresent exemplary embodiment, the display control unit 133 superimposesa panning main guide 813, the tilting main guide 814, and the meridianlines 722 onto the captured image, and then outputs the resultant imageto the display apparatus 170.

Referring to FIG. 8A, a display image 810 is displayed on the displayapparatus 170. An imaging center 811 is an intersection of the opticalaxis of the optical system for forming a subject image on the imagesensor and a plane containing the captured image 810 captured by theimaging unit 111. A panning rotation center 812 denotes the position ofa panning rotation center in a plane containing the captured image 810.

The panning main guide 813 is a panning direction guide for indicatingthe panning moving direction such that the panning main guide 813overlaps a moving path to be traced by the imaging center 811 while theimaging unit 111 is driven for panning. The tilting main guide 814passes through the imaging center 811 to indicate the tilting drivedirection. The latitude lines 721 and the meridian lines 722 are set upon the spherical surface 720. In the present exemplary embodiment, thepanning main guide 813 and the tilting main guide 814 are superimposedonto the captured image 810, and then the latitude lines 721 and themeridian lines 722 are superimposed thereon to auxiliarily indicate thepanning and tilting drive directions. The present exemplary embodimentcontrols display positions of the latitude lines 721 and the meridianlines 722 by using setting values of the latitude lines 721 and themeridian lines 722 set up on the spherical surface 720, and the imagingposition of the imaging apparatus 110.

A method for displaying the panning main guide 813 is similar to therelevant method in the first exemplary embodiment illustrated in FIG.3E, and duplicated explanation will be omitted. A method for displayingthe latitude lines 721 and the meridian lines 722 will be describedbelow. Based on imaging position information (panning angle, tiltingangle, amount of rotation, etc.), the viewing angle, and the image sizereceived from the imaging apparatus 110, the display control unit 133 ofthe client apparatus 130 sets up a virtual display image 730 on avirtual spherical surface 720, as illustrated in FIG. 7B. Then, thedisplay control unit 133 projects preset latitude lines 721 and meridianlines 722 onto the display image 730 from the imaging apparatus position710. Thus, the latitude lines 721 and the meridian lines 722 projectedonto the display image 730 are displayed as the latitude lines 721 andthe meridian lines 722 on the display image 810.

The above-mentioned method is to be considered as an example, anddisplaying the latitude lines 721 and the meridian lines 722 is basednot only thereon. Although the latitude lines 721 and the meridian lines722 are drawn by dashed lines in FIG. 8A, they may be drawn by solidlines or arrows. Alternatively, the latitude lines 721, the meridianlines 722, the panning main guide 813, and the tilting main guide 814may be displayed in different colors for distinction.

Suppose that, in the state illustrated in FIG. 8A, the imaging unit 111is driven for tilting and the imaging direction 208 of the imaging unit111 coincides with the panning rotational axis 203. A display image 820in this state will be described below with reference to FIG. 8B. In thisstate, similar to the first exemplary embodiment, the panning main guide813 converges to a point and can no longer be displayed. However, thelatitude lines 721 set up on the spherical surface 720 auxiliarilyindicates the panning drive direction, enabling the user to recognizethe panning drive direction. Further, the latitude lines 721 enable theuser to confirm in advance how the panning main guide 813 will changewhen the imaging unit 111 is driven for panning. Thus, the latitudelines 721 in the present exemplary embodiment play the roles of thepanning sub guides 315 and 316 in the first exemplary embodiment.

Although the above-mentioned latitude lines 721 and the meridian lines722 may be constantly displayed similar to the first exemplaryembodiment, displaying these lines is based not only on theabove-mentioned method. The latitude lines 721 and the meridian lines722 may be displayed only when the imaging direction 208 falls within aspecific range specified by a separately set threshold value, or set upto be displayed or hidden by the user. Further, the latitude lines 721and meridian lines 722 may be set to variable positions on the sphericalsurface 720.

When the imaging unit 111 is subsequently driven for rotation, thedisplay image 810 in FIG. 8A rotates around the imaging center 811similar to the first exemplary embodiment. With the rotation of thecaptured image 810, the display control unit 133 rotates the orientationof the panning main guide 813 and the tilting main guide 814 around theimaging center 811. Then, the display control unit 133 newly sets up thedisplay image 730 on the spherical surface 720 depending on the amountof rotation drive operation by the rotation drive unit 114. Then, thedisplay control unit 133 projects the latitude lines 721 and themeridian lines 722 after rotation onto the display image 730, anddisplays the projected latitude lines 721 and the meridian lines 722 onthe display image 730 after rotation. Alternatively, together with thepanning main guide 813 and the tilting main guide 814, the latitudelines 721 and the meridian lines 722 may be rotated around the imagingcenter 811 of the display image 810.

Thus, the client apparatus 130 can show the user the panning and tiltingdrive directions when the imaging unit 111 is driven for rotation.Further, even when the panning rotation center 812 coincides with theimaging center 811 and the panning main guide 813 converges to a pointand can no longer be displayed, the latitude lines 721 can show the userthe panning drive direction. Then, the latitude lines 721 can show theuser in advance how the panning main guide 813 will change when theimaging unit 111 is driven for tilting toward the panning rotationcenter 812 (in the upward direction in FIG. 8A).

Operations of the client apparatus 130 and the imaging apparatus 110 inthe imaging control system according to the present exemplary embodimentare similar to those in the first exemplary embodiment illustrated inFIGS. 5 and 6, and duplicated explanation will be omitted.

With the above-mentioned configuration, the latitude lines 721 are setup on the virtual spherical surface 720 set up around the imagingapparatus 110, and the latitude lines 721 are superimposed onto thedisplay image 810. Therefore, even when the imaging center 811 overlapsthe panning rotation center 812 and the panning main guide 813 is nolonger displayed, the latitude lines 721 enable the user to recognizethe moving direction. The latitude lines 721 also enable the user torecognize how the panning main guide 813 will change before the imagingcenter 811 overlaps the panning rotation center 812. Thus, the presentexemplary embodiment can suitable notify the user of the movingdirection of the imaging area. Therefore, the user can recognize thepanning drive direction of the imaging apparatus 110 more easily thanthe case where only the panning main guide 813 is displayed on thedisplay image 810.

Although the present exemplary embodiment displays the panning mainguide 813 indicating the panning moving direction such that the panningmain guide 813 overlaps a moving path to be traced by the imaging center811, the effect of the present invention can be achieved even bydisplaying only the latitude lines 721 without displaying the panningmain guide 813.

A third exemplary embodiment of the present invention will be describedbelow based on a case where an icon is displayed on the panning mainguide 313 or the panning sub guide 315. This icon is used to input acommand for changing the imaging direction 208 to the panning direction.

A configuration of the imaging control system according to the presentexemplary embodiment is illustrated in FIG. 10. Elements similar tothose in the first exemplary embodiment illustrated in FIG. 1 areassigned the same reference numerals, and duplicated explanation will beomitted.

In the present exemplary embodiment, the display control unit 133 of theclient apparatus 130 superimposes the above-mentioned icon for inputtinga command to the panning operation unit 138 onto a position indicatingthe panning drive direction on the captured image captured by theimaging unit 111. Lines indicating the panning drive direction refer,for example, to the panning main guide 313 and the panning sub guide 315illustrated in FIGS. 3A to 3F. In the present exemplary embodiment, thepanning sub guide 315 indicates that a point existing on a straight linepassing through the panning rotation center 312 and the imaging center311, where the distance thereof from the panning rotation center 312 islonger than the distance from the panning rotation center 312 to theimaging center 311, will rotate around the panning rotation center 312.In this case, the panning main guide 313 or the panning sub guide 315does not need to be actually superimposed onto the captured image. Theicon is preferably superimposed onto a position where the panning mainguide 313 or the panning sub guide 315 should be superimposed.

In the present exemplary embodiment, the user interface control unit 136includes a generation unit 141 for generating panning operation icons(hereinafter referred to as panning operation buttons) for inputtingcommands to the panning operation unit 138, and tilting operation icons(hereinafter referred to as tilting operation buttons) for inputtingcommands to the tilting operation unit 139. The CPU 137 loads an icondisplay program from the memory 132 and executes it to control thegeneration unit 141 to generate these icons. Then, the UI control unit136 further includes a pointer unit 142 for instructing the displaycontrol unit 133 to display a pointer on the display apparatus 170,according to a command from the input apparatus 150 (described below)such as a mouse.

When the UI control unit 136 detects that the user clicks the mouse on apanning operation button displayed on the display apparatus 170, thepanning operation unit 138 transmits a relevant command for controllingthe panning drive unit 112 to the imaging apparatus 110. Then, when theUI control unit 136 detects that the user clicks the mouse on a tiltingoperation button displayed on the display apparatus 170, the tiltingoperation unit 139 transmits a relevant command for controlling thetilting drive unit 113 to the imaging apparatus 110. The user clicks oneach operation button by using the input apparatus 150. The operation ofeach operation button is not limited to mouse operation using thepointer. The user may operate each operation button by touching it on atouch panel. Operating the operation buttons is based not only on theabove-mentioned method but also on any other method as long as the usercan input commands to the panning operation unit 138 by using theoperation buttons. The panning and tilting operation buttons aresuperimposed onto positions indicating the panning and tilting drivedirections, respectively, on the captured image. Displaying theseoperation buttons in this way enables the user to perform panning andtilting drive operations in a more intuitive way.

FIGS. 11A to 11D illustrate exemplary display images displayed on thedisplay apparatus 170 through display control by the display controlunit 133. FIG. 11A illustrates a display image 1110 produced when theimaging direction 208 is close to the perpendicular to the panningrotational axis 203. Referring to FIG. 11A, the display image 1110 isdisplayed on the display apparatus 170 after the display control unit133 applies display control to the captured image received from theimaging apparatus 110 by the client apparatus 130.

The display control unit 133 displays panning operation buttons 317 andtilting operation buttons 318 on the display image 1110. The panningoperation buttons 317 are icons used by the user to drive the imagingunit 111 in the panning direction. In the state illustrated in FIG. 11A,the panning operation buttons 317 are displayed at both ends of thepanning main guide 313. The tilting operation buttons 318 are icons usedby the user to drive the imaging unit 111 in the tilting direction. Inthe state illustrated in FIG. 11A, the tilting operation buttons 318 aredisplayed at both ends of the tilting direction guide 314. Of the twopanning operation buttons 317, the right-hand side panning operationbutton 317 is an icon for instructing the panning drive operation in therightward direction, and the left-hand side panning operation button 317is an icon for instructing the panning drive operation in the leftwarddirection. Of the two tilting operation buttons 318, the top tiltingoperation button 318 is an icon for instructing the tilting driveoperation in the upward direction, and the bottom tilting operationbutton 318 is an icon for instructing the tilting drive operation in thedownward direction. Displaying the panning operation buttons 317 on thepanning direction guide 313 and the tilting operation buttons 318 on thetilting direction guide 314 enables the user to recognize the drivedirection and perform panning and tilting drive operations in a moreintuitive way.

Suppose that, in the state illustrated in FIG. 11A, the user instructsfrom the input apparatus 150 to drive the imaging unit 111 for tilting,the imaging unit 111 is driven for tilting toward the panning rotationcenter 312 (in the upward direction in FIG. 11A), and the imagingdirection 208 comes close to the panning rotational axis 203. A displayimage 1120 in this state will be described below with reference to FIG.11B.

When the imaging direction 208 comes close to the panning rotationalaxis 203, the one panning operation button 317 displayed at one end ofthe panning main guide 313 overlaps the other panning operation button317 displayed at the other end of the panning main guide 313. However,when display positions of the panning operation buttons 317 are changedto both ends of the panning sub guide 315, the one panning operationbutton 317 does not overlap the other panning operation button 317.Specifically, when an angle between the panning rotational axis 203 andthe optical axis of the imaging unit 111 is greater than a predeterminedthreshold value, the display control unit 133 displays the panningoperation buttons 317 on the panning main guide 313. When the anglebetween the panning rotational axis 203 and the optical axis of theimaging unit 111 becomes equal to or less than the predeterminedthreshold value, the display control unit 133 changes display positionsof the panning operation buttons 317. A method for setting the thresholdvalue will be described below. Thus, the display control unit 133superimposes the panning operation buttons 317 onto positions where thedistance from the panning rotation center 312 to the panning operationbuttons 317 is longer than the distance from the panning rotation center312 to the imaging center 311 on the captured image.

A method for determining the threshold value will be described below.For example, the value of the angle 303 with which the distance from thepanning rotation center 312 to the imaging center 311 is a quarter ofthe length of the tilting direction guide 314 displayed on the displayimage 1110 can be set as the threshold value. Specifically, when acircle around the panning rotation center 312 and having a radius equalto the distance between the imaging center 311 and the panning rotationcenter 312 is entirely contained in the display image 1110, displaypositions of the panning operation buttons 317 may be changed.

Referring to FIG. 3E, when the panning rotation center 312 is anintersection of a plane containing the captured image 302 and thepanning rotational axis 203, a distance x between the imaging center 311and the panning rotation center 312 is represented by formula (3).x=L·tan β  (3)where L denotes the magnitude of the vector 301 and β denotes the valueof the angle 303.

Therefore, an angle β with which the value of L·tanβ is smaller than aquarter of the length of the tilting direction guide 314 displayed onthe display image 1110 can be set as the threshold value of the angle303. Setting the threshold value is based not only on theabove-mentioned method but may be based on other methods. For example,an angle β with which the distance x is equal to or less than a totalsum of a quarter of the length of the tilting direction guide 314displayed on the display image 1110 and a distance between the panningmain guide 313 and the panning sub guide 315 displayed thereon can beset as the threshold value of the angle 303.

The above-mentioned method can prevent the one panning operation button317 displayed at one end of the panning direction guide 313 fromoverlapping the other panning operation button 317 displayed at theother end of the panning direction guide 313.

Suppose that, in the state illustrated in FIG. 11B, the imaging unit 111is further driven for tilting toward the panning rotation center 312,and the imaging center 311 coincides with the panning rotational axis203. A display image 1130 in this state will be described below withreference to FIG. 11C. In this state, since the position of the panningrotation center 312 derived from state values of the imaging apparatus110 coincides with the imaging center 311, the panning main guide 313converges to a point and can no longer be displayed. However, thepanning sub guide 315 set up to be displayed on the outer side of thepanning main guide 313 is kept being displayed without converging to apoint, enabling the user to recognize the panning drive direction. Whenthe imaging center 311 coincides with the panning rotational axis 203,the panning operation buttons 317 set to be displayed at both ends ofthe panning main guide 313 are no longer displayed on the display image1130. However, displaying the panning operation buttons 317 at both endsof the panning sub guide 315 prevents the panning operation buttons 317from being hidden as the imaging unit 111 is driven for tilting. Thus,the display control unit 133 displays the panning operation buttons 317on the panning direction guide 315 indicating that changing the imagingdirection 208 to the panning direction rotates the captured image aroundthe imaging center 311.

In the present exemplary embodiment, with the rotation of a displayimage 1140, the display control unit 133 rotates display positions ofthe panning operation buttons 317 and the tilting operation buttons 318around the imaging center 311 and then superimposes these operationbuttons onto the captured image, as illustrated in FIG. 11D.Specifically, the panning operation buttons 317 are displayed at bothends of the panning direction guide 313 on the rotated captured image410 illustrated in FIG. 4. Further, the tilting operation buttons 318are displayed at both ends of the tilting direction guide 314 in therotated captured image 410 illustrated in FIG. 4. Thus, the clientapparatus 130 can display the panning operation buttons 317 and thetilting operation buttons 318 in the panning and tilting directions,respectively, on the captured image produced when the imaging unit 111is driven for rotation. Further, even when the panning rotation center312 coincides with the imaging center 311, and the panning main guide313 converges to a point and can no longer be displayed, the panning subguide 315 can show the user the panning drive direction. Further, evenwhen the panning rotation center 312 coincides with the imaging center311, and the panning main guide 313 converges to a point and can nolonger be displayed, displaying the panning operation buttons 317 on thepanning sub guide 315 enables the user to use the panning operationbuttons 317.

Operations of the client apparatus 130 and the imaging apparatus 110 inthe imaging control system according to the present exemplary embodimentwill be described below. In the present exemplary embodiment, in stepS110 illustrated in FIG. 5, the client apparatus 130 displays thepanning operation buttons 317 on the panning main guide 313 or thepanning sub guides 315 and 316 as mentioned above. Other operations ofthe client apparatus 130 and the imaging apparatus 110 according to thepresent exemplary embodiment are similar to those in the first exemplaryembodiment illustrated in FIGS. 5 and 6, and duplicated explanation willbe omitted.

According to the above-mentioned configuration, the display control unit133 superimposes the panning operation buttons 317 onto a panningdirection guide not overlapping a moving path to be traced by theimaging center 311 while the imaging unit 111 is driven for panning.Thus, the imaging control system according to the present exemplaryembodiment can suitably display the panning operation icons even whenthe optical axis of the imaging apparatus 110 comes close to the panningrotational axis 203. Therefore, even when the imaging center 311overlaps the panning rotation center 312 and the panning main guide 313is no longer displayed, the user can drive the imaging unit 111 forpanning by using the panning operation buttons 317.

Although, in the present exemplary embodiment, the display control unit133 changes display positions of the panning operation buttons 317 whenthe value of the angle 303 between the panning rotational axis 203 andthe optical axis of the imaging unit 111 is equal to or less than apredetermined threshold value, the display positions of the panningoperation buttons 317 may remain unchanged. Specifically, the panningoperation buttons 317 may be displayed on the panning sub guide 315illustrated in FIGS. 11A to 11D regardless of the value of the angle303.

Although the present exemplary embodiment displays the panning mainguide 313 and the panning sub guides 315 and 316, only the panningoperation buttons 317 may be displayed without displaying the panningmain guide 313 and the panning sub guides 315 and 316. When only thepanning operation buttons 317 are displayed, some arrangements may bemade to make it easier for the user to recognize the panning drivedirection. Specifically, arrows may be drawn on the panning operationbuttons 317 to indicate the panning drive direction or triangularpanning operation buttons 317 may be displayed to indicate the panningdrive direction with their apexes. Thus, the user can recognize thepanning drive direction in a display image and perform a panning driveoperation in an intuitive way.

A fourth exemplary embodiment of the present invention will be describedbelow. The present exemplary embodiment displays panning operationbuttons at predetermined positions which are symmetrical with respect toa straight line passing through an imaging center and indicating thetilting direction on a display image.

An imaging control system according to the fourth exemplary embodimentis implemented with elements similar to those of the imaging controlsystem according to the third exemplary embodiment illustrated in FIG.10, and duplicated explanation will be omitted. Elements similar tothose in the first exemplary embodiment illustrated in FIG. 10 areassigned the same reference numerals.

In the fourth exemplary embodiment, a display image 1210 displayed onthe display apparatus 170 through display control by the display controlunit 133 will be described below with reference to FIGS. 12A to 12C.FIG. 12A illustrates the display image 1210 produced when the imagingdirection 208 is close to the perpendicular to the panning rotationalaxis 203. An imaging center 1211 is an intersection of the optical axisof the optical system for forming a subject image on the image sensorand a plane containing the captured image captured by the imaging unit111. A panning rotation center 1212 indicates the position of a panningrotation center in a plane containing the display image 1210. A methodfor setting the panning rotation center 1212 is similar to the relevantmethod in the first exemplary embodiment illustrated in FIG. 3E, andduplicated explanation will be omitted.

Referring to FIGS. 12A to 12C, a panning main guide 1213 is a panningdirection guide for indicating the panning direction on the displayimage 1210. The panning main guide 1213 is displayed as a circle or arcaround the panning rotation center 1212 passing through panningoperation buttons 1217 or as a straight line passing therethrough. Inthe present exemplary embodiment, the panning main guide 1213 issuperimposed onto the captured image such that the panning main guide1213 does not pass through the imaging center 1211 and the distance fromthe panning rotation center 1212 to the panning main guide 1213 islonger than the distance from the panning rotation center 1212 to theimaging center 1211. In the present exemplary embodiment, the panningmain guide 1213 indicates that a predetermined point existing on astraight line passing through the panning rotation center 1212 and theimaging center 1211, where the distance from the panning rotation center1212 is longer than the distance thereof from the panning rotationcenter 1212 to the imaging center 1211, will rotate around the panningrotation center 1212. A tilting main guide 1214 is a tilting directionguide for indicating the tilting moving direction such that the tiltingmain guide 1214 overlaps a moving path to be traced by the imagingcenter 1211 while the imaging unit 111 is driven for tilting.

Then, the display control unit 133 superimposes the panning operationbuttons 1217 and tilting operation buttons 1218 onto the display image1210. The panning operation buttons 1217 are icons used for inputtingcommands to the panning operation unit 138. In the state illustrated inFIG. 12A, the panning operation buttons 1217 are superimposed ontopredetermined positions on the display image 1210. The tilting operationbuttons 1218 are icons for inputting commands to the tilting operationunit 139. In the state illustrated in FIG. 3A, the tilting operationbuttons 1218 are displayed at both ends of the tilting main guide 1214.

Display positions of the panning operation buttons 1217 on the displayimage 1210 will be described below. The two panning operation buttons1217 illustrated in FIG. 12A are displayed line-symmetrically withrespect to the tilting main guide 1214 to be equally distant from thepanning rotation center 1212. Thus, the two panning operation buttons1217 are equally distant from any point on the straight line of thetilting main guide 1214. Therefore, the two panning operation buttons1217 can be displayed on a circle around the panning rotation center1212 existing on the straight line of the tilting main guide 1214. Then,the distance from the panning rotation center 1212 to the panningoperation buttons 1217 is set up to be longer than the distance from thepanning rotation center 1212 to the imaging center 1211.

Suppose that, in the state illustrated in FIG. 12A, the user instructsthe input apparatus 150 to drive the imaging unit 111 for tilting, theimaging unit 111 is driven for tilting toward the panning rotationcenter 1212 (in the upward direction in FIG. 12A), and the imagingdirection 208 comes close to the panning rotational axis 203. Thedisplay image 1210 in this state will be described below with referenceto FIG. 12B. As the imaging direction 208 comes close to the panningrotational axis 203, the imaging center 1211 and the panning rotationcenter 1212 come close to each other. On the other hand, since displaypositions of the panning operation buttons 1217 remain unchanged afterthe tilting drive operation is made, the panning main guide 1213maintains a suitable radius without degrading visibility. Suppose that,in the state illustrated in FIG. 12B, the imaging unit 111 is furtherdriven for tilting toward the panning rotation center 1212 and then theimaging direction 208 coincides with the panning rotational axis 203.The display image 1210 in this state is illustrated in FIG. 12C. In thisstate, the panning rotation center 1212 derived from state values of thenetwork camera coincides with the imaging center 1211. Since displaypositions of the panning operation buttons 1217 remain unchanged in thisstate, the panning main guide 1213 maintains at least a certain fixedradius, enabling an operator to confirm the drive direction and operatethe panning operation buttons 1217.

FIGS. 13A and 13B illustrate exemplary display images produced when therotation mechanism is driven. FIG. 13A illustrates a display image 1310produced when the rotation mechanism is driven when the imagingdirection 208 is close to the perpendicular to the panning rotationalaxis 203. The display control unit 133 performs display control so thatthe panning operation buttons 1217 and the tilting operation buttons1218 are rotated with the drive operation of the rotation mechanism. Thepanning main guide 1213 and the tilting main guide 1214 are also rotatedaccordingly. Suppose that, in the state illustrated in FIG. 13A, theimaging unit 111 is driven for tilting toward the panning rotationalcenter 1212 and the imaging direction 208 coincides with the panningrotational axis 203. A display image 1320 in this state is illustratedin FIG. 13B. In this state, since display positions of the panningoperation buttons 1217 remain unchanged after the tilting operation ismade, the panning main guide 1213 maintains a certain fixed radius,enabling the operator to operate each mechanism by using the operationbuttons without degrading visibility. Thus, even when the optical axisof the imaging apparatus 110 comes close to the panning rotational axis203, the imaging control system according to the present exemplaryembodiment can suitably display the panning operation icons.

Operations of the imaging control system according to the presentexemplary embodiment are similar to those of the imaging control systemillustrated in FIGS. 5 and 6, and duplicated explanation will beomitted.

The imaging control system according to the fourth exemplary embodimentdisplays the panning operation buttons 1217 at predetermined positionson the display image 1310 or 1320 such that the distance from thepanning rotational center 1212 to the panning operation buttons 1217 islonger than the distance from the panning rotation center 1212 to theimaging center 1211 while the imaging unit 111 is driven for panning andtilting. With the imaging control system according to the fourthexemplary embodiment, the panning main guide 1213 is superimposed ontothe captured image such that the panning main guide 1213 constantlypasses through the panning operation buttons 1217. Therefore, even whenthe optical axis of the imaging apparatus 110 comes close to the panningrotational axis 203, the panning operation icons can be displayed.

The first to fourth exemplary embodiments have been specificallydescribed based on a case where the imaging apparatus 110 is providedwith the rotation mechanism. In a case where the imaging apparatus 110is not provided with the rotation mechanism, virtual rotation can beperformed by rotating the image on the client apparatus 130. The presentinvention can be implemented even with such a configuration. Even inthis case, similar to a case where the imaging unit 111 is driven forrotation, the imaging apparatus 110 can show the user the panning andtilting drive directions even when the optical axis of the imagingapparatus 110 comes close to the panning rotational axis 203.

Although the first to fourth exemplary embodiments display a tiltingmain guide in addition to a panning main guide and panning sub guides,only the panning sub guides may be displayed. Even when only the panningsub guides are displayed, the first to fourth exemplary embodiments cansuitably show the user the moving direction of the imaging area withoutlosing a panning direction guide when the imaging unit 111 is driven fortilting. The client apparatus 130 described in the first to fourthexemplary embodiments may be integrated with the imaging apparatus 110.Specifically, the present invention can be implemented even after theclient apparatus 130 superimposes the panning main guide 313 and thepanning sub guide 315 onto the captured image captured by the imagingapparatus 110 and then transmits a display image to the displayapparatus 170 via the network 190.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment (s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Applications No.2010-197840 filed Sep. 3, 2010 and No. 2010-198775 filed Sep. 6, 2010,which are hereby incorporated by reference herein in their entirety.

What is claimed is:
 1. A control apparatus for controlling an imagingapparatus capable of changing an imaging direction of an imaging unit inpanning and tilting directions, the control apparatus comprising: anacquisition unit configured to acquire a captured image captured by theimaging unit; and a display control unit configured to control todisplay, on the captured image, if an imaging direction of the imagingunit does not correspond to a panning rotational axis for rotating theimaging unit in the panning direction, a first panning direction guidefor indicating that the imaging direction changes along a circle arounda rotation center by panning the imaging unit, and to control todisplay, on the captured image, if the imaging direction of the imagingunit corresponds to the panning rotational axis, a second panningdirection guide for indicating that the imaging unit rotates around animaging center which is an intersection of an optical axis of theimaging unit and a plane containing the captured image by panning theimaging unit.
 2. The control apparatus according to claim 1, furthercomprising: a panning operation unit configured to instruct the imagingapparatus to change the imaging direction in the panning direction; anda display control unit configured to display on the panning directionguide an icon for inputting a command to the panning operation unit. 3.The control apparatus according to claim 2, wherein the display controlunit superimposes a plurality of the icons at predetermined positions onthe captured image which are symmetrical with respect to a straight linepassing through the imaging center and indicating the tilting direction,and superimposes on the captured image the panning direction guidepassing through the icons.
 4. The control apparatus according to claim1, wherein the display control unit is configured to control to display,if the imaging direction of the imaging unit corresponds to the panningrotational axis, the second panning direction guide indicating acaptured image rotates along a circle around the imaging center bypanning the imaging unit.
 5. The control apparatus according to claim 1,wherein the display control unit is configured to control to display, ifthe imaging direction of the imaging unit corresponds to the panningrotational axis, the second panning direction guide instead ofdisplaying the first panning direction guide.
 6. A control apparatus forcontrolling an imaging apparatus capable of changing an imagingdirection of an imaging unit in panning and tilting directions, thecontrol apparatus comprising: an acquisition unit configured to acquirea captured image captured by the imaging unit; and a display controlunit configured to control to display on the captured image a panningdirection guide for indicating the panning direction which changes incurvature corresponding to the tilting direction, by superimposing thepanning direction guide onto the captured image which is captured by theimaging unit pointing one imaging direction such that the panningdirection guide does not pass through an imaging center which is anintersection of an optical axis of the imaging unit and a planecontaining the captured image.
 7. The control apparatus according toclaim 6, wherein the display control unit is configured to superimposethe panning direction guide onto the captured image in accordance with achange in the imaging direction in a rotation direction in which theimaging direction is rotated around the optical axis.
 8. The controlapparatus according to claim 6, wherein the display control unit isconfigured to set up latitude lines on a virtual spherical surfacearound the imaging apparatus and to superimpose, as the panningdirection guide, the latitude lines onto the captured image arranged onthe virtual spherical surface.
 9. The control apparatus according toclaim 6, wherein the display control unit is configured to superimposethe panning direction guide onto the captured image when an anglebetween the panning rotational axis and the optical axis is equal to orless than a predetermined threshold value.
 10. The control apparatusaccording to claim 6, wherein the display control unit is configured tosuperimpose the panning direction guide onto the captured image when anangle between the panning rotational axis and the optical axis is equalto or less than a threshold value which changes with a viewing angle ofthe imaging unit.
 11. A control apparatus for controlling an imagingapparatus capable of changing an imaging direction of an imaging unit inpanning and tilting directions, the control apparatus comprising: apanning operation unit configured to instruct the imaging apparatus tochange the imaging direction in the panning direction; an acquisitionunit configured to acquire a captured image captured by the imagingunit; and a display control unit configured to display, when anintersection of a panning rotational axis for rotating the imagingdirection of the imaging unit in the panning direction and a planecontaining the captured image is set as a panning rotation center and anintersection of an optical axis of the imaging unit and the plane is setas an imaging center, a panning direction guide on the captured imageand display on the panning direction guide an icon for inputting acommand to the panning operation unit, the panning direction guideindicating a part or the whole of a circle around the panning rotationcenter passing through a predetermined point existing on a straight linepassing through the panning rotation center and the imaging center,where a distance of the predetermined point from the panning rotationcenter is longer than a distance from the panning rotation center to theimaging center.
 12. The control apparatus according to claim 11,wherein, when an angle between the panning rotational axis and theoptical axis is equal to or less than a predetermined threshold value,the display control unit superimposes the icon onto the captured imagesuch that a distance from the panning rotation center to the icon islonger than the distance from the panning rotation center to the imagingcenter.
 13. An imaging control system comprising an imaging apparatusand a control unit for controlling the imaging apparatus, wherein theimaging apparatus comprises: an imaging unit configured to capture animage; a panning change unit configured to move the imaging unit in apanning direction; and a tilting change unit configured to move theimaging unit in a tilting direction, and wherein the control unitcomprises: an acquisition unit configured to acquire a captured imagecaptured by the imaging unit; and a display control unit configured tocontrol to display, on the captured image, if an imaging direction ofthe imaging unit does not correspond to a panning rotational axis forrotating the imaging unit in the panning direction, a first panningdirection guide for indicating that the imaging direction changes alonga circle around a rotation center by panning the imaging unit, and tocontrol to display, on the captured image, if an imaging direction ofthe imaging unit corresponds to the panning rotational axis, a secondpanning direction guide for indicating that the imaging unit rotatesaround an intersection of an optical axis of the imaging unit and aplane containing the captured image by panning the imaging unit.
 14. Amethod for controlling an imaging apparatus capable of changing animaging direction of an imaging unit in panning and tilting directions,the method comprising: acquiring a captured image captured by theimaging unit; and controlling to display, on the captured image, if animaging direction of the imaging unit does not correspond to a panningrotational axis for rotating the imaging unit in the panning direction,a first panning direction guide for indicating that the imagingdirection changes along a circle around a rotation center by panning theimaging unit, and to control to display, on the captured image, if theimaging direction of the imaging unit corresponds to the panningrotational axis, a second panning direction guide for indicating thatthe imaging unit rotates around an intersection of an optical axis ofthe imaging unit and a plane containing the captured image by panningthe imaging unit.
 15. A method for controlling an imaging apparatuscapable of changing an imaging direction of an imaging unit in panningand tilting directions, the method comprising: acquiring a capturedimage captured by the imaging unit; and controlling to display on thecaptured image the panning direction which changes in curvaturecorresponding to the tilting direction by superimposing the panningdirection guide onto the captured image which is captured by the imagingunit pointing one imaging direction such that the panning directionguide does not pass through an imaging center which is an intersectionof an optical axis of the imaging unit and a plane containing thecaptured image.
 16. The method according to claim 15, whereincontrolling to display the panning direction guide by superimposing thepanning direction guide onto the captured image when an angle betweenthe panning rotational axis and the optical axis is equal to or lessthan a predetermined threshold value.
 17. The method according to claim15, wherein controlling to display the panning direction guide bysuperimposing the panning direction guide onto the captured image whenan angle between the panning rotational axis and the optical axis isequal to or less than a threshold value which changes with a viewingangle of the imaging unit.
 18. A computer-readable storage mediumstoring a program for causing a control apparatus for controlling animaging apparatus capable of changing an imaging direction of an imagingunit in panning and tilting directions to execute a method comprising:acquiring a captured image captured by the imaging unit; and controllingto display, on the captured image, if an imaging direction of theimaging unit does not correspond to a panning rotational axis forrotating the imaging unit in the panning direction, a first panningdirection guide for indicating that the imaging direction changes alonga circle around a rotation center by panning the imaging unit, and tocontrol to display, on the captured image, if the imaging direction ofthe imaging unit corresponds to the panning rotational axis, a secondpanning direction guide for indicating that the imaging unit rotatesaround an intersection of an optical axis of the imaging unit and aplane containing the captured image by panning the imaging unit.
 19. Anon-transitory computer-readable storage medium storing a program forcausing a control apparatus for controlling an imaging apparatus capableof changing an imaging direction of an imaging unit in panning andtilting directions to execute a method comprising: acquiring a capturedimage captured by the imaging unit; and controlling to display on thecaptured image the panning direction which changes in curvaturecorresponding to the tilting direction by superimposing the panningdirection guide onto the captured image which is captured by the imagingunit pointing one imaging direction such that the panning directionguide does not pass through an imaging center which is an intersectionof an optical axis of the imaging unit and a plane containing thecaptured image.
 20. The storage medium according to claim 19, whereinthe method includes superimposing the panning direction guide onto thecaptured image when an angle between the panning rotational axis and theoptical axis is equal to or less than a predetermined threshold value.21. The storage medium according to claim 19, wherein the methodincludes superimposing the panning direction guide onto the capturedimage when an angle between the panning rotational axis and the opticalaxis is equal to or less than a threshold value which changes with aviewing angle of the imaging unit.